Microphone gain using a time of flight (ToF) laser range finding system

ABSTRACT

Range to a human speaker is determined using a laser-based time of flight (ToF) system, with the range then being used to adjust the gain of a microphone receiving the speaker&#39;s voice. If desired, an acoustic-based Direction of Arrival (DoA) system uses acoustic information to determine the direction of incoming sound, such as a person talking, and the direction of the sound is then used to focus the area of laser illumination.

FIELD

The present application relates to technically inventive, non-routinesolutions that are necessarily rooted in computer technology and thatproduce concrete technical improvements.

BACKGROUND

When multiple computerized devices interact with each other at closerange, they may employ sensors such as cameras and laser range findersto map their environment. As understood herein, the use of such sensorsby multiple devices can cause mutual interference due to overlappingemissions of laser illumination.

More generally, in laser range-finding systems that use time of flight(ToF) principles to find distances from an origin to various objects,for a comprehensive mapping of the environment the laser must be sweptleft and right and up and down to detect objects in the area anddetermine their ranges from the origin.

Relatedly, a difficulty in a voice recognition system is to correctlyset the audio gain. If the gain is too high, distortion results or toomuch ambient sounds are amplified. If the gain is too low, desiredsounds will not be detected.

SUMMARY

As understood herein, instead of solely relying on the voice recognitionsystem to adapt the gain level by trying a setting, detecting speech andthen either raising or lowering the gain in an iterative process, arange-finding laser is used to determine the distance to the speaker.The range is then used to establish the gain.

Accordingly, a device includes at least one computer medium that is nota transitory signal and that in turn includes instructions executable byat least one processor to receive at least a first acoustic signal froma source of sound, and to determine at least an azimuthal direction tothe source of sound from an origin based at least in part on the firstacoustic signal. The instructions are executable to transmit at least afirst range-finding beam along the azimuthal direction determined basedat least in part on the first acoustic signal to render a distance tothe source of sound. The instructions are further executable to, basedat least in part on the distance rendered using the range-finding beam,adjust a gain of an amplifier of an audio system.

The gain of the amplifier may be linearly correlated to the distance.Or, the gain of the amplifier may be logarithmically correlated to thedistance.

The distance can be rendered based on intraocular spacing between aperson's pupils as determined from a 3D map generated based onreflections of the laser range-finding beam. Or, the distance can berendered using the laser range-finding beam is based on a time of flight(ToF) determination.

The instructions may be executable to determine, based on a texture ofat least one surface, a sound absorption characteristic of the surface.The instructions may be executable to adjust the gain of the amplifierbased at least in part on the sound absorption characteristic of thesurface.

In another aspect, an assembly includes at least one laser, at least onereceiver configured to output signals representative of reflections oflight emitted by the laser, and at least one computer storage. At leastone processor is configured to access instructions on the computerstorage to determine a distance to an object based at least in part onthe signals from the receiver. The instructions are executable toestablish a gain of at least one audio amplifier based at least in parton the distance.

The audio amplifier can be a microphone amplifier for communicating withthe processor for receiving and amplifying voice signals from at leastone microphone. In addition or alternatively, the audio amplifier can bea speaker amplifier for communicating with the processor for outputtingsignals to at least one audio speaker.

In another aspect, a method includes transmitting laser range-findingbeams toward a person, and determining, using reflections of the beams,a distance to the person. The method includes using the distance toadjust a gain of an audio amplifier.

The details of the present application, both as to its structure andoperation, can be best understood in reference to the accompanyingdrawings, in which like reference numerals refer to like parts, and inwhich:

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example device;

FIG. 2 is a block diagram of a system showing two devices, it beingunderstood that more than two devices may be used;

FIG. 3 is a schematic top view of two devices showing field of view ofthe devices;

FIG. 4 is a flow chart of example general logic according to presentprinciples;

FIG. 5 is a flow chart of example logic for establishing a peer deviceas a master device;

FIG. 6 is a flow chart of example alternate logic for establishing apeer device as a master device;

FIG. 7 is a schematic diagram of an example transmission schedule;

FIG. 8 is an example schematic plan view of multiple devicesillustrating a space division interference-avoidance technique;

FIG. 9 is a flow chart of example logic attendant to the technique inFIG. 8;

FIG. 10 is a schematic plan view of a computerized range-finding andmapping device using sound from an object to direct a laser range findertoward the object;

FIG. 11 is a flow chart of logic consistent with present principles;

FIG. 12 is a block diagram of an example computerized range-findingdevice with an audio amplifier that has its gain adjusted based on therange to the person speaking as determined by the laser-based ToFsystem; and

FIG. 13 is a flow chart consistent with present principles.

DETAILED DESCRIPTION

This disclosure relates generally to computer ecosystems includingaspects of multiple computerized devices. A system herein includingcomputerized devices may include server and client components, connectedover a network such that data may be exchanged between the client andserver components. The client components may include one or morecomputing devices such as portable televisions (e.g. smart TVs,Internet-enabled TVs), portable computers such as laptops and tabletcomputers, and other mobile devices including smart phones andadditional examples discussed below. These client devices may operatewith a variety of operating environments. For example, some of theclient computers may employ, as examples, operating systems. Theseoperating environments may be used to execute one or more browsingprograms that can access web applications hosted by the Internet serversdiscussed below.

Servers may include one or more processors executing instructions thatconfigure the servers to receive and transmit data over a network suchas the Internet. Or, a client and server can be connected over a localintranet or a virtual private network.

Information may be exchanged over a network between the clients andservers. To this end and for security, servers and/or clients caninclude firewalls, load balancers, temporary storages, and proxies, andother network infrastructure for reliability and security. One or moreservers may form an apparatus that implement methods of providing asecure community including but not limited to social networks to networkmembers.

As used herein, instructions refer to computer-implemented steps forprocessing information in the system. Instructions can be implemented insoftware, firmware or hardware and include any type of programmed stepundertaken by components of the system.

A processor may be any conventional general purpose single- ormulti-chip processor that can execute logic by means of various linessuch as address lines, data lines, and control lines and registers andshift registers. A processor may be implemented by a digital signalprocessor (DSP), for example.

Software modules described by way of the flow charts and user interfacesherein can include various sub-routines, procedures, etc. Withoutlimiting the disclosure, logic stated to be executed by a particularmodule can be redistributed to other software modules and/or combinedtogether in a single module and/or made available in a shareablelibrary.

Present principles described herein can be implemented as hardware,software, firmware, or combinations thereof; hence, illustrativecomponents, blocks, modules, circuits, and steps are set forth in termsof their functionality.

Further to what has been alluded to above, logical blocks, modules, andcircuits described below can be implemented or performed with a generalpurpose processor, a digital signal processor (DSP), a fieldprogrammable gate array (FPGA) or other programmable logic device suchas an application specific integrated circuit (ASIC), discrete gate ortransistor logic, discrete hardware components, or any combinationthereof designed to perform the functions described herein. A processorcan be implemented by a controller or state machine or a combination ofcomputing devices.

The functions and methods described below, when implemented in software,can be written in an appropriate language such as but not limited to C#or C++, and can be stored on or transmitted through a computer-readablestorage medium such as a random access memory (RAM), read-only memory(ROM), electrically erasable programmable read-only memory (EEPROM),compact disk read-only memory (CD-ROM) or other optical disk storagesuch as digital versatile disc (DVD), magnetic disk storage or othermagnetic storage devices including removable thumb drives, etc. Aconnection may establish a computer-readable medium. Such connectionscan include, as examples, hard-wired cables including fiber optic andcoaxial wires and digital subscriber line (DSL) and twisted pair wires.

Components included in one embodiment can be used in other embodimentsin any appropriate combination. For example, any of the variouscomponents described herein and/or depicted in the Figures may becombined, interchanged or excluded from other embodiments.

“A system having at least one of A, B, and C” (likewise “a system havingat least one of A, B, or C” and “a system having at least one of A, B,C”) includes systems that have A alone, B alone, C alone, A and Btogether, A and C together, B and C together, and/or A, B, and Ctogether, etc.

Now specifically referring to FIG. 1, an example system 10 is shown,which may include one or more of the example devices mentioned above anddescribed further below in accordance with present principles. The firstof the example devices included in the system 10 is an examplecomputerized device 12.

The computerized device 12 may be an Android®-based system. Thecomputerized device 12 alternatively may also include a computerizedInternet enabled (“smart”) telephone, a tablet computer, a notebookcomputer, a wearable computerized device such as e.g. computerizedInternet-enabled watch, a computerized Internet-enabled bracelet, othercomputerized Internet-enabled devices, a computerized Internet-enabledmusic player, computerized Internet-enabled head phones, a computerizedInternet-enabled implantable device such as an implantable skin device,etc. Regardless, it is to be understood that the computerized device 12and/or other computers described herein is configured to undertakepresent principles (e.g. communicate with other CE devices to undertakepresent principles, execute the logic described herein, and perform anyother functions and/or operations described herein).

Accordingly, to undertake such principles the computerized device 12 canbe established by some or all of the components shown in FIG. 1. Forexample, the computerized device 12 can include one or more displays 14that may be implemented by a high definition or ultra-high definition“4K” or higher flat screen and that may or may not be touch-enabled forreceiving user input signals via touches on the display. Thecomputerized device 12 may include one or more speakers 16 foroutputting audio in accordance with present principles, and at least oneadditional input device 18 such as e.g. an audio receiver/microphone forreceiving input sound including but not limited to audible commands tothe computerized device 12 to control the computerized device 12. Theexample computerized device 12 may also include one or more networkinterfaces 20 for communication over at least one network 22 such as theInternet, a WAN, a LAN, a PAN etc. under control of one or moreprocessors 24. Thus, the interface 20 may be, without limitation, aWi-Fi transceiver, which is an example of a wireless computer networkinterface, such as but not limited to a mesh network transceiver. Theinterface 20 may be, without limitation a Bluetooth transceiver, Zigbeetransceiver, IrDA transceiver, Wireless USB transceiver, wired USB,wired LAN, Powerline or MoCA. It is to be understood that the processor24 controls the computerized device 12 to undertake present principles,including the other elements of the computerized device 12 describedherein such as e.g. controlling the display 14 to present images thereonand receiving input therefrom. Furthermore, note the network interface20 may be, e.g., a wired or wireless modem or router, or otherappropriate interface such as, e.g., a wireless telephony transceiver,or Wi-Fi transceiver as mentioned above, etc.

In addition to the foregoing, the computerized device 12 may alsoinclude one or more input ports 26 such as, e.g., a high definitionmultimedia interface (HDMI) port or a USB port to physically connect(e.g. using a wired connection) to another CE device and/or a headphoneport to connect headphones to the computerized device 12 forpresentation of audio from the computerized device 12 to a user throughthe headphones. For example, the input port 26 may be connected via wireor wirelessly to a cable or satellite source 26 a of audio videocontent. Thus, the source 26 a may be, e.g., a separate or integratedset top box, or a satellite receiver. Or, the source 26 a may be a gameconsole or disk player containing content that might be regarded by auser as a favorite for channel assignation purposes described furtherbelow.

The computerized device 12 may further include one or more computermemories 28 such as disk-based or solid state storage that are nottransitory signals, in some cases embodied in the chassis of the deviceas standalone devices or as a personal video recording device (PVR) orvideo disk player either internal or external to the chassis of thedevice for playing back AV programs or as removable memory media. Alsoin some embodiments, the computerized device 12 can include a positionor location receiver such as but not limited to a cellphone receiver,GPS receiver and/or altimeter 30 that is configured to e.g. receivegeographic position information from at least one satellite or cellphonetower and provide the information to the processor 24 and/or determinean altitude at which the computerized device 12 is disposed inconjunction with the processor 24. However, it is to be understood thatthat another suitable position receiver other than a cellphone receiver,GPS receiver and/or altimeter may be used in accordance with presentprinciples to e.g. determine the location of the computerized device 12in e.g. all three dimensions.

In some embodiments the computerized device 12 may include one or morecameras 32 that may be, e.g., a thermal imaging camera, a digital camerasuch as a webcam, and/or a camera integrated into the computerizeddevice 12 and controllable by the processor 24 to gather pictures/imagesand/or video in accordance with present principles. Also included on thecomputerized device 12 may be a Bluetooth transceiver 34 and other NearField Communication (NFC) element 36 for communication with otherdevices using Bluetooth and/or NFC technology, respectively. An exampleNFC element can be a radio frequency identification (RFID) element.

Further still, the computerized device 12 may include one or moreauxiliary sensors 37 (e.g., a motion sensor such as an accelerometer,gyroscope, cyclometer, or a magnetic sensor, an infrared (IR) sensor forreceiving IR commands or other signals from a remote control or laser,an optical sensor, a speed and/or cadence sensor, a gesture sensor (e.g.for sensing gesture command), etc.) providing input to the processor 24.The computerized device 12 may include one or more motors 38, which maybe a battery-powered motor, and one or more actuators 39 coupled to themotor 38 and configured to cause the device 12 to ambulate. In oneexample, the actuator 39 is a simple axle-and-wheel actuator that can beenergized by the motor 38 to cause the device 12 to roll across asurface. In another example the actuator 39 may include one or morelinear actuators with joints to cause the device 12 to move in arobotic, walking-type fashion on multiple legs. These are but twoexamples of motion actuators that can be included in the device 12.

In addition to the foregoing, it is noted that the computerized device12 may also include an infrared (IR) transmitter and/or IR receiverand/or IR transceiver 42 such as a laser or an IR data association(IRDA) device. A battery (not shown) may be provided for powering thecomputerized device 12.

Still referring to FIG. 1, in addition to the computerized device 12,the system 10 may include one or more other computer device types thatmay include some or all of the components shown for the computerizeddevice 12 and that may wirelessly communicate with the device 12 tocontrol it. In one example, a first device 44 and a second device 46 areshown and may include similar components as some or all of thecomponents of the computerized device 12. Fewer or greater devices maybe used than shown.

In the example shown, to illustrate present principles all three devices12, 44, 46 are assumed to be members of a local network in, e.g., adwelling 48, illustrated by dashed lines.

The example non-limiting first device 44 may include one or moretouch-sensitive surfaces 50 such as a touch-enabled video display forreceiving user input signals via touches on the display. The firstdevice 44 may include one or more speakers 52 for outputting audio inaccordance with present principles, and at least one additional inputdevice 54 such as e.g. an audio receiver/microphone for e.g. enteringaudible commands to the first device 44 to control the device 44. Theexample first device 44 may also include one or more network interfaces56 for communication over the network 22 under control of one or moreprocessors 58. Thus, the interface 56 may be, without limitation, aWi-Fi transceiver, which is an example of a wireless computer networkinterface, including mesh network interfaces. It is to be understoodthat the processor 58 controls the first device 44 to undertake presentprinciples, including the other elements of the first device 44described herein such as e.g. controlling the display 50 to presentimages thereon and receiving input therefrom. Furthermore, note thenetwork interface 56 may be, e.g., a wired or wireless modem or router,or other appropriate interface such as, e.g., a wireless telephonytransceiver, or Wi-Fi transceiver as mentioned above, etc.

In addition to the foregoing, the first device 44 may also include oneor more input ports 60 such as, e.g., a HDMI port or a USB port tophysically connect (e.g. using a wired connection) to another computerdevice and/or a headphone port to connect headphones to the first device44 for presentation of audio from the first device 44 to a user throughthe headphones. The first device 44 may further include one or moretangible computer readable storage medium 62 such as disk-based or solidstate storage. Also in some embodiments, the first device 44 can includea position or location receiver such as but not limited to a cellphoneand/or GPS receiver and/or altimeter 64 that is configured to e.g.receive geographic position information from at least one satelliteand/or cell tower, using triangulation, and provide the information tothe device processor 58 and/or determine an altitude at which the firstdevice 44 is disposed in conjunction with the device processor 58.However, it is to be understood that that another suitable positionreceiver other than a cellphone and/or GPS receiver and/or altimeter maybe used in accordance with present principles to e.g. determine thelocation of the first device 44 in e.g. all three dimensions.

Continuing the description of the first device 44, in some embodimentsthe first device 44 may include one or more cameras 66 that may be,e.g., a thermal imaging camera, a digital camera such as a webcam, etc.Also included on the first device 44 may be a Bluetooth transceiver 68and other Near Field Communication (NFC) element 70 for communicationwith other devices using Bluetooth and/or NFC technology, respectively.An example NFC element can be a radio frequency identification (RFID)element.

Further still, the first device 44 may include one or more auxiliarysensors 72 (e.g., a motion sensor such as an accelerometer, gyroscope,cyclometer, or a magnetic sensor, an infrared (IR) sensor, an opticalsensor, a speed and/or cadence sensor, a gesture sensor (e.g. forsensing gesture command), etc.) providing input to the CE deviceprocessor 58. The first device 44 may include still other sensors suchas e.g. one or more climate sensors 74 (e.g. barometers, humiditysensors, wind sensors, light sensors, temperature sensors, etc.) and/orone or more biometric sensors 76 providing input to the device processor58. In addition to the foregoing, it is noted that in some embodimentsthe first device 44 may also include an infrared (IR) transmitter and/orIR receiver and/or IR transceiver 42 such as a laser or an IR dataassociation (IRDA) device. A battery may be provided for powering thefirst device 44. The device 44 may communicate with the computerizeddevice 12 through any of the above-described communication modes andrelated components.

The second device 46 may include some or all of the components describedabove.

Now in reference to the afore-mentioned at least one server 80, itincludes at least one server processor 82, at least one computer memory84 such as disk-based or solid state storage, and at least one networkinterface 86 that, under control of the server processor 82, allows forcommunication with the other devices of FIG. 1 over the network 22, andindeed may facilitate communication between servers, controllers, andclient devices in accordance with present principles. Note that thenetwork interface 86 may be, e.g., a wired or wireless modem or router,Wi-Fi transceiver, or other appropriate interface such as, e.g., awireless telephony transceiver.

Accordingly, in some embodiments the server 80 may be an Internetserver, and may include and perform “cloud” functions such that thedevices of the system 10 may access a “cloud” environment via the server80 in example embodiments. Or, the server 80 may be implemented by agame console or other computer in the same room as the other devicesshown in FIG. 1 or nearby.

Before proceeding to example embodiments in FIGS. 2-9, in which multiplecomputerized devices map each other in their respective areas, it is tobe understood that the illustrated system is but an example of oneenvironment in which the acoustic-based direction of arrival (DoA)principles discussed herein to aim time of flight (ToF) based laserrange finding systems may be used. The acoustic-based DoA principlesdiscussed herein to aim ToF-based laser range finding systems may beused in other embodiments as well, such as acoustic/ultrasonic rangefinding systems and camera-based range finding systems that might use,e.g., phase detection.

FIG. 2 shows that multiple devices 12, 12A may be controlled byrespective CE devices 44, 46 to interact on a surface 200 such as a flatplanar surface.

FIG. 3 shows that the first device 12 may have a camera providing afield of view (FOV) with a FOV angle 300. The first device 12 may emitlaser range-finding light such as IR light along one or more rangefinding axes 302. The camera may be implemented by a complementary metaloxide semiconductor (CMOS) camera that can detect both visible andinfrared light so as to be able to produce still or video images alongwith detections of laser reflections for purposes of generating a depthmap.

Likewise, the second device 12A may have a camera providing a field ofview (FOV) with a FOV angle 304. The first device 12 may emit laserrange-finding light such as IR light along one or more range findingaxes 306. More than two devices may be used. In the example shown, eachdevice is within the FOV of the other device, and the FOVs of thedevices overlap as shown. The devices 12, 12A emitting their respectivelaser beams establish an optical micro-mesh.

FIG. 4 shows example general logic. Commencing at state 400, internalcomputer clocks of the devices 12, 12A are synchronized with each other.This can be done by synchronizing the clocks to a common “heartbeat”such as a master clock that may be the clock of any of the masterdevices discussed below, or other clock.

At block 402 a master device is established. Examples of how this may bedone are discussed below. The master device may be a system server thatcommunicates with the devices, or a controller such as the CE device 44,or one of the peer devices 12, 12A. The matter device may assign timeslots in a transmission schedule to each device at block 404, and theneach device activates its emission (such as a laser range-findingemission) at block 406 only in a period defined by one of its assignedslots in the schedule.

FIG. 5 illustrates example logic when one of the peer devices 12, 12A isto be a master. A first one of the devices (“A”) sends a negotiationsignal at block 500 to the other devices, time-stamping the transmissiontime in a “let's establish a master” message carried in the signal. Thesignal may be sent via Wi-Fi, Bluetooth, laser, or other transmissionmeans. Other devices (“B, . . . N”) may also send time-stampednegotiation signals at block 502. At block 504, the devices access thetimestamps of their messages and the messages of the other devices, andthe device whose message carries the earliest time stamp can beacknowledged, by each device, as the master device at block 504.

Alternatively, at block 600 in FIG. 6 each device 12, 12A . . . can sendidentification information such serial number and/or model number and/ormanufacturer name to other devices. At block 602 the device with itsidentification information matching a predetermined information can beacknowledged, by each device, as the master device.

FIG. 7 shows an example transmission schedule that is divided intosequential time slots each of which may have the same temporal length asthe other slots, or a different length than the other slots. These slotscan be assigned to respective devices at block 404 in FIG. 4 by themaster device. It is to be understood that the transmission schedule maystart at an initial time defined by the system “heartbeat” and that thetime slots repeat, so that each device is typically assigned multipletime slots separated from each other by the time slots of the otherdevices.

FIG. 8 illustrates that in lieu of (or in addition to) using timedivision multiplexing (TDM) as in FIGS. 4-7 to alleviate mutualinterference due to overlapping emissions of laser illumination, spacedivision (SD) may be used to alleviate mutual interference. In FIG. 8,as an example, three devices 12, 12A, and 12B are assigned respectivespatial zones A, B, and C. The zones may be static, with the devices 12,12A, and 12B being restricted to remaining within their zones. Or, thezones may be dynamic and may change as the devices move, e.g., on acommon flat surface. As an example of the latter, suppose the distancealong the y-axis between the devices 12A and 12B is D1, with themidpoint being at 800. A horizontal boundary represented by the dashedline may be established by a perpendicular to the distance vector thatpasses through the midpoint 800.

Similarly, if the distance D2 along the x-axis between the devices 12and 12A is D2 with midpoint 802, a vertical boundary represented by thedashed line may be established by a perpendicular to the distance vectorthat passes through the midpoint 802. The zones A, B, and C are definedby the boundaries. Because the distance D3 along the x-axis between thedevices 12 and 12B is greater than the distance D2 between the devices12 and 12A, its midpoint 804 is not used to define a boundary. In otherimplementations the midpoint of the greater distance may be used. Otherpoints along the distance vector than the midpoint may be used. In anycase, it will be appreciated that as a device moves, the boundariesdefining the zones may also move and, hence, the sizes of the zones canchange dynamically.

FIG. 9 illustrates to block 900 that the zones shown in FIG. 8 areestablished and remain static or change dynamically as disclosed. Thezones may be established by, e.g., a master device such as any of theabove-described master devices. At block 902 when the devices can move,each device is constrained to remain within its assigned spatial zone.

FIGS. 10 and 11 illustrate additional features that may be used with thetechnology in FIGS. 1-9 or in other systems to efficiently conductlaser-based ToF range-finding to objects. A computerized device 1000that may be implemented by any appropriate one of the devices describedpreviously and that may include any of the internal components describedpreviously can include one or more microphones 1002. The device 1000includes at least one and in the example shown, plural microphones 1002arranged in an array. In one embodiment at least one microphone 1002 maybe a directional microphone such as a so-called “shotgun” microphonethat directly outputs an indication of the direction in which receivedsound was detected. In other embodiments a multi-microphone array isused. In the example, three microphones 1002 are arranged at vertices ofa triangle. In other embodiments, two and only two microphones 1002 maybe provided. When two and only two microphones 1002 are provided, theymay be spaced laterally from each other in the x-y (azimuthal) plane andmay be offset from each other in the z-dimension (elevationaldimension). The microphones 1002 may be ultrasonic microphones in someembodiments.

The microphones 1002 may communicate via wired and/or wireless pathswith at least one processor 1004 accessing instructions on a computerstorage 1006. The processor 1004 may control one or more lasers 1008such as infrared (IR) lasers to emit light toward objects, withreflections of the light from the objects being detected by a receiver1010 that communicates with the processor 1004. Based on the timedifference between transmission of the laser light and receipt of thereturn reflection and speed of light, the processor 1004 can determinethe range to objects reflecting the laser light.

Thus, one or more objects 1012 may be irradiated by the laser light andmay reflect the light back to the receiver 1010. The objects 1012 mayemit sound. The objects 1012 may be people, in which case the sound theyemit may be spoken voices, or other computerized devices, in which casethey typically include one or more speakers 1014 for emitting sound. Insome embodiments, the speakers 1014 may be ultrasonic (US) speakers, sothat the acoustic-based direction of arrival (DoA) principles discussedbelow may be imperceptible to humans. However, human-perceptible soundmay also be used.

FIG. 11 illustrates example logic for using an acoustic-based DoA systemsuch as the example system shown in FIG. 10 to determine the directionof incoming sound, such as from an object 1012 talking or emitting soundfrom its speakers 1014, to focus a laser-based ToF system such as theexample system shown in FIG. 10 to narrow the area of laserillumination, thereby improving the signal to noise ratio. The DoAsystem may also provide elevation information pertaining to the sourceof the sound, to further narrow the required field of view of the laserToF system.

With more specificity, at block 1100 the object 1012 emits sound. Whenthe object is a person the person may emit sound by speaking. When theobject is a computerized device, the object may emit sound by emitting asound such as a US pulse or other sonic frequency sound pattern on itsspeaker(s) 1014. In the context of the non-limiting system of FIGS. 2-9,if desired the object 1012 may emit a sonic pattern according to aschedule known to all the computerized devices, so that all the devicesknow the identity of the emitting object. In addition or alternatively,the object may emit sound including a sonic code that uniquelyidentifies the object.

Moving to block 1102, the device 1000 detects the sound emitted at block1100. Proceeding to block 1104, based on the detected sound, the device1000 (e.g., the processor 1004) determines the direction along which thesound was received and, hence, the direction of the object 1012 relativeto the device 1000, which essentially establishes an origin.

To do this, the device 1000 may simply note the indication of sounddirection from the microphone 1002 when the microphone 1002 is adirectional microphone. When an array of omnidirectional microphones isused, the processor 1004 may note the delays between times of receptionof the sound at the various microphones and converting the delays todistances using the relationship between the speed of sound, time, anddistance. The distances can then be triangulated to determine theazimuthal direction to the object 1012 from the device 1100, andfurthermore when microphones in the array are elevationally offset fromeach other, the elevational direction of the object 1012 may also bedetermined.

Once having determined the direction of the object 1012 in azimuth and,if desired, in elevation as well using acoustics as described above, thelogic moves to block 1106 to direct the light from the laser 1008 in thedirection determined at block 1104. To do this, the laser 1008 can bemounted on a gimbal or other movable mount which is controlled by theprocessor 1004 to turn to the direction determined at block 1104, withlaser range-finding light being emitted along the direction. Or, thelaser 1008 may be stationarily mounted on the device 1000 and theprocessor 1004 may cause the entire device to turn as appropriate to aimthe laser 1008 along the direction determined at block 1004. At block1108, using ToF principles discussed previously, the device determinesthe range to the object 1012 based on the time difference between lasertransmission and laser reflection reception, also mapping the object1012 in relative space if desired using the laser-derived range fromblock 1108 and sonic-derived direction to the object from block 1104.Note that to conserve power, the laser 1008 may not be activated unlessand until a sonic-derived direction to an object 1012 is determined andused to aim the laser at the object.

FIGS. 12 and 13 illustrate the use of laser range-finding to adjust thegain of one or more audio amplifiers. In FIG. 12, a computerized device1200 that may be implemented by any appropriate one of the devicesdescribed previously and that may include any of the internal componentsdescribed previously can include one or more microphones 1202. Thedevice 1200 includes at least one and in some embodiments pluralmicrophones 12002 arranged in an array. In one embodiment at least onemicrophone 12002 may be a directional microphone such as a so-called“shotgun” microphone that directly outputs an indication of thedirection in which received sound was detected. In other embodiments amulti-microphone array is used. In an example, three microphones 1202may be arranged at vertices of a triangle. In other embodiments, two andonly two microphones 1202 may be provided. When two and only twomicrophones 1202 are provided, they may be spaced laterally from eachother in the x-y (azimuthal) plane and may be offset from each other inthe z-dimension (elevational dimension). The microphones 1202 may beultrasonic microphones in some embodiments.

The microphones 1202 may communicate via wired and/or wireless pathswith at least one processor 1204 accessing instructions on a computerstorage 1206. The processor 1204 may control one or more lasers 1208such as infrared (IR) lasers to emit light toward objects such a person1209 who may be speaking, with reflections of the light from the objectbeing detected by a receiver 1210 that communicates with the processor1204. Based on the time difference between transmission of the laserlight and receipt of the return reflection and speed of light, theprocessor 1204 can determine the range or distance “D” to the person1209 reflecting the laser light. The distance “D” is thus a distancebetween the irradiated object and a reference origin, in this case, thereceiver 1210, which for present purposes can be taken to represent thedistance between the person 1209 and the microphones 1202.

Thus, one or more objects may be irradiated by the laser light and mayreflect the light back to the receiver 1210. The objects 1209 may bepeople, in which case the sound they emit may be spoken voices, or theirradiated object may be other computerized devices.

In the example embodiment shown, signals from the microphone 1202 may beamplified by one or more microphone amplifiers 1212, which may in turnsend the amplified signals to the processor 1204 and receive automaticgain control (AGC) signals from the processor 1204 over one or morecommunication paths that cause the amplifier 1212 to adjust its gain upor down. Likewise, one or more speaker amplifiers 1214 may be providedthat receive signals from the processor 1204 over one or morecommunication paths and output amplified signals to one or more speakers1216 associated with the device 1200 (e.g., by being on or incommunication with the device 1200). In some examples, in addition to orin lieu of the laser range-finding system, the device 1200 may includeone or more image sensors 1218 such as a two dimensional imagerimplemented by a complementary metal-oxide-semiconductor (CMOS) deviceor charge-coupled device (CCD).

Now referring to FIG. 13, example operation of the device 1200 shown inFIG. 12 is explained. Commencing at block 1300, in some non-limitingembodiments the acoustic-based DoA principles discussed above may beused to determine the direction of the person 1209. Proceeding to block1302, using the direction provided by the acoustic-based DoA system, thelaser 1208 may be aimed as described above toward the person 1209 andactivated to cause a reflection from the person 1209 to be received bythe receiver 1210.

Moving to block 1304, the processor 1204 may next determine the distance“D” to the person 1209 as described above using the laser-based ToFprinciples already discussed. Then, at block 1306 the processor 1204 mayadjust or otherwise establish the gain of either one or both of themicrophone amplifier 1212 or speaker amplifier 1214 based on thedistance “D” determined at block 1304.

In some examples, the amplifier gain adjustment(s) may be linearlycorrelated to the distance “D”. For instance, suppose the gain of theamplifier to be adjusted as at some nominal reference gain_(ref) whenthe distance “D” is equal to a reference distance D_(ref). The gain at anon-reference distance “D” may be determined by, for example:Gain=gain_(ref)(D/D _(ref))

In other examples, the amplifier gain adjustment(s) may belogarithmically correlated to the distance “D”, for example:Gain=gain_(ref) 10 log D/D _(ref)

Table lookups correlating various values of “D” and “gain” may be usedin lieu of the above algorithms for speed, if desired, with the tableentry of the “D” nearest the laser-measured “D” being used to extractthe corresponding gain.

Thus, the processor's ability to execute voice recognition on speechfrom the person 1209 for example is improved. Instead of adapting thegain level by trying a setting, detecting speech and the either raisingor lowering the gain in an iterative process, the above-describedlaser-based ToF principles (augmented if desired by the above-describedDoA principles to aim the laser) can be used to provide ancillaryinformation.

Additionally, not only can the ToF system determine range to the person,but as described previously it can generate a 3D map of the person'sface, which may be input to a face recognition algorithm to determinethe identity of the person and/or identify that the person being imagedis speaking, thereby confirming that the distance to the person isindeed that which should be used to adjust the audio amplifier gain.

As an alternative to determining the distance “D” using ToFcalculations, the laser-based ToF system may simply determine thespeakers' distance by determining, from the 3D map, the angle betweenthe pupils, intraocular spacing being relatively standard amongsthumans, and correlate the intraocular spacing to a distance to theperson. If desired, the 2D image sensor 1218 may be used for this samepurpose.

The resolution of the 3D TOF system and/or 2D camera system may be greatenough to detect surface textures of the objects in the room. Based uponthe type of surface texture, the approximate sound absorptive propertiesof the object/surface can be determined using, e.g., a table look-upcorrelating surface texture types with acoustic absorption properties.For example, texture types that may be discerned from a high resolution3D TOF system and/or 2D camera system may be whether a ceiling isacoustic tile, or a floor is a shag carpet or hardwood, or a wall iswood or wall paper or tile.

Based upon the acoustical properties of the various textured surfaces inthe room, it may be determined whether the room is acousticallyreflective or absorptive. The acoustical absorption properties ofobjects in the path between the microphone and person also can be usedto adjust the gain. As an example, if a room's surfaces and objects areprimarily acoustically reflective, gain can be reduced for better echocancellation, whereas if a room's surfaces and objects are primarilyabsorptive of sound, gain can be increased to better amplify theperson's voice.

While particular techniques and machines are herein shown and describedin detail, it is to be understood that the subject matter which isencompassed by the present invention is limited only by the claims.

What is claimed is:
 1. A device comprising: at least one computer mediumthat is not a transitory signal and that comprises instructionsexecutable by at least one processor to: receive from at least onmicrophone at least a first acoustic signal from a source of sound;determine at least an azimuthal direction to the source of sound from anorigin based at least in part on the first acoustic signal; transmit atleast a first range-finding beam along the azimuthal directiondetermined based at least in part on the first acoustic signal to rendera distance to the source of sound; based at least in part on thedistance rendered using the range-finding beam, adjust a gain of anamplifier of an audio system; identify a sound absorption characteristicof a surface; and adjust the gain of the amplifier based at least inpart on the sound absorption characteristic of the surface.
 2. Thedevice of claim 1, comprising the at least one processor.
 3. The deviceof claim 1, wherein the gain of the amplifier is linearly correlated tothe distance.
 4. The device of claim 1, wherein the gain of theamplifier is logarithmically correlated to the distance.
 5. The deviceof claim 1, wherein the distance rendered using the laser range-findingbeam is based on intraocular spacing between a person's pupils asdetermined from a 3D map generated based on reflections of the laserrange-finding beam.
 6. The device of claim 1, wherein identifying thesound absorption characteristic of the surface comprises: determining,based on a texture of the surface, the sound absorption characteristicof the surface.
 7. The device of claim 1, wherein the instructions areexecutable to determine at least an elevational direction to the sourceof sound from the origin based at least in part on the first acousticsignal.
 8. The device of claim 1, wherein the instructions areexecutable to determine a range to the object based on a time oftransmission of the first laser range-finding beam and a time ofreception of a reflection of the first laser range-finding beam.
 9. Anassembly comprising: at least one laser; at least one receiverconfigured to output signals representative of reflections of lightemitted by the laser; at least one computer storage; and at least oneprocessor configured to access instructions on the computer storage to:receive at least a first acoustic signal from the object; determine atleast an azimuthal direction to the object from an origin based at leastin part on the first acoustic signal; transmit at least a first laserrange-finding beam along the azimuthal direction; determine a distanceto an object based at least in part on the signals from the receiver;and establish a gain of at least one audio amplifier based at least inpart on the distance and a sound absorption characteristic of theobject.
 10. The assembly of claim 9, wherein the audio amplifier is amicrophone amplifier for communicating with the processor for receivingand amplifying voice signals from at least one microphone.
 11. Theassembly of claim 9, wherein the audio amplifier is a speaker amplifierfor communicating with the processor for outputting signals to at leastone audio speaker.
 12. The assembly of claim 9, wherein the distancerendered based on intraocular spacing between a person's pupils asdetermined from a 3D map generated based on reflections of light emittedby the laser.
 13. The assembly of claim 9, wherein the distance is basedon a time of flight (ToF) determination.
 14. A computer-implementedmethod comprising: transmitting laser range-finding beams toward aperson, a direction to the person being identified least in part ondetecting a signal from the person representing the direction to theperson; determining, using reflections of the beams, a distance to theperson; identifying a sound absorption characteristic of the person; andusing the distance and sound absorption characteristic, adjusting a gainof an audio amplifier.
 15. The method of claim 14, comprising:determining at least an azimuthal direction to the object based on soundreceived from the object; and aiming the laser range-finding beamsaccording to the azimuthal direction.
 16. The method of claim 14,wherein the audio amplifier is a microphone amplifier for receiving andamplifying voice signals from at least one microphone.
 17. The method ofclaim 14, wherein the audio amplifier is a speaker amplifier foroutputting signals to at least one audio speaker.
 18. The method ofclaim 14, comprising linearly correlating the gain of the audioamplifier to the distance.
 19. The method of claim 14, comprisinglogarithmically correlating the gain of the audio amplifier to thedistance.